Papers with probabilistic model
Probabilistic FastText for Multi-Sense Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Probabilistic FastText model for word embeddings captures word senses, sub-word structure, and uncertainty information. |
| Approach: | They propose a model for word embeddings that captures multiple word senses . they represent each word with a Gaussian mixture density, with each vector representing an n-gram . |
| Outcome: | The proposed model outperforms dictionary-level probabilistic embeddings on word-similarity benchmarks. |
Unsupervised Morphology Learning with Statistical Paradigms (C18-1)
Copied to clipboard
| Challenge: | Existing models treat words as concatenation of morphemes, but some use transformations like rewrite rules to recognize dependencies between morphs. |
| Approach: | They propose an unsupervised model that exploits the notion of paradigms for morphological segmentation that can be applied to a homogeneous set of words. |
| Outcome: | The proposed model significantly improves on the Morpho-Challenge dataset in English, Turkish, and Finnish. |
A Natural Bias for Language Generation Models (2023.acl-short)
Copied to clipboard
| Challenge: | a standard probabilistic model for language generation has likely not yet learnt many semantic or syntactic rules of natural language, making it difficult to estimate the probability distribution over next tokens. |
| Approach: | They propose to initialise bias terms in a model's final linear layer with the log-unigram distribution and use it to output the unigram frequency statistics as prior knowledge. |
| Outcome: | The proposed method improves learning efficiency and improves overall performance. |
Quality Estimation for Partially Subjective Classification Tasks via Crowdsourcing (2020.lrec-1)
Copied to clipboard
| Challenge: | a common approach to quality estimation is to ask multiple reviewers to evaluate the same artifacts. |
| Approach: | They propose a probabilistic model for subjective classification tasks that incorporates the qualities of artifacts as well as the abilities and biases of creators and reviewers as latent variables to be jointly inferred. |
| Outcome: | The proposed model estimates the quality of speech more effectively than a vote aggregation, measured by correlation with a fine-grained classification by experts. |
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering (2021.tacl-1)
Copied to clipboard
| Challenge: | Recent studies have shown that language models capture different types of knowledge regarding facts or commonsense knowledge. |
| Approach: | They examine how language models can be calibrated to make their confidence scores correlate better with the likelihood of correctness. |
| Outcome: | The proposed calibration methods improve confidence scores on QA tasks and improve accuracy. |
Learning Logic Rules for Document-Level Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for document-level relation extraction relied on implicitly powerful representations, which makes the model less transparent. |
| Approach: | They propose a probabilistic model for document-level relation extraction by learning logic rules. |
| Outcome: | The proposed model outperforms baseline models in relation performance and logical consistency. |
The problem with probabilistic DAG automata for semantic graphs (N19-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) annotations are directed acyclic graphs, but most probabilistic models view them as strings or trees. |
| Approach: | They show that some DAG automata cannot be made into useful probabilistic models by assigning weights to transitions. |
| Outcome: | The proposed model can't be made into useful probabilistic models by assigning weights to transitions . the proposed model is not feasible for all variants, but it is problematic for planar variants if they are not rooted . |
Phone Inventories and Recognition for Every Language (2022.lrec-1)
Copied to clipboard
| Challenge: | Identifying phone inventories is crucial component in language documentation and preservation of endangered languages. |
| Approach: | They propose a probabilistic and non-probabilistic phone inventory model that estimates the phone inventory for any language listed in Glottolog. |
| Outcome: | The proposed model outperforms baseline models by 6.5 F1 and improves the PER (phone error rate) in phone recognition by 25%. |
A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation (2025.naacl-long)
Copied to clipboard
| Challenge: | Current large language models (LLMs) produce factually incorrect statements . |
| Approach: | They propose a probabilistic framework for LLM hallucination detection that generates a belief tree by expanding a statement into logically related claims and reasoning globally about the relationships between these claims. |
| Outcome: | The proposed method improves on multiple hallucination detection benchmarks by 3%-9% over state-of-the-art models. |
Valency-Augmented Dependency Parsing (D18-1)
Copied to clipboard
| Challenge: | valency analysis is a complex task that requires a large number of subcategorizations, such as the number and types of syntactic dependents. |
| Approach: | They propose a parsing approach that explicitly models the number and types of syntactic dependents as valency patterns and a probabilistic model for tagging them. |
| Outcome: | The proposed approach outperforms the state-of-the-art labeled attachment score on 53 treebanks representing 41 languages and outperformed the previous state- of-the art labeles by 0.7. |
pEBR: A Probabilistic Approach to Embedding Based Retrieval (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing embedding-based retrieval systems rely on heuristic and suboptimal cutoffs for item retrieval. |
| Approach: | They propose a probabilistic Embedding-Based Retrieval framework that learns a shared semantic representation space for both queries and items. |
| Outcome: | The proposed framework improves retrieval precision and recall, and ablation studies show it captures the differences between head-to-tail queries. |
A Probabilistic Model for Joint Learning of Word Embeddings from Texts and Images (D18-1)
Copied to clipboard
| Challenge: | Existing approaches combine language and perception to infer word embeddings . however, the embeddables produced by such models do not reflect the actual word representations. |
| Approach: | They propose a probabilistic model that integrates linguistic and perceptual inputs to explain observed word-context pairs in a text corpus. |
| Outcome: | The proposed model achieves competitive or stronger results on tasks of assessing pairwise word similarity and image/caption retrieval compared to other state-of-the-art models. |
Modelling Instance-Level Annotator Reliability for Natural Language Labelling Tasks (N19-1)
Copied to clipboard
| Challenge: | Existing models that estimate annotators' reliability only consider binary labels and multi-class labels. |
| Approach: | They propose an unsupervised model which can handle binary and multi-class labels and integrate neural networks to model the dependency between latent variables and instances. |
| Outcome: | The proposed model can handle binary and multi-class labels and can estimate reliability of annotators across instances. |
Max-Margin Incremental CCG Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | a new incremental parser reduces the number of beam search violations and minimises the biggest violation. |
| Approach: | They propose to use beam search optimisation to minimise all beam search violations instead of minimising only the biggest violation. |
| Outcome: | The proposed parser outperforms existing non-incremental parsers and minimises all beam search violations instead of minimising the biggest violation. |
Making Heads and Tails of Models with Marginal Calibration for Sparse Tagsets (2021.findings-emnlp)
Copied to clipboard
| Challenge: | despite high accuracy, modern neural networks can still suffer from severe miscalibration. |
| Approach: | They propose to use tag frequency grouping to measure calibration error in different frequency bands to reduce error. |
| Outcome: | The proposed techniques reduce calibration error across the marginal distribution for two existing sequence taggers. |
Probabilistic Case-based Reasoning for Open-World Knowledge Graph Completion (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for learning non-parametric representations of entities and relations are based on tensor factorization or sophisticated neural approaches. |
| Approach: | They propose a case-based reasoning system that retrieves ‘cases’ that are similar to the given problem and then stores them in its parameters. |
| Outcome: | The proposed model outperforms state-of-the-art methods on several benchmark datasets and is non-parametric and grows dynamically as new entities and relations arrive in the KB. |
Enhancing Unsupervised Generative Dependency Parser with Contextual Information (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to unsupervised dependency parsing are based on probabilistic generative models that learn the joint distribution of the given sentence and its parse. |
| Approach: | They propose a probabilistic model that generates a sentence and its parse from a latent representation, which encodes global contextual information of the generated sentence. |
| Outcome: | The proposed model achieves competitive accuracy compared with state-of-the-art models. |
Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming (P19-1)
Copied to clipboard
| Challenge: | Discrete structures are ubiquitous in the study of natural languages, for example in morphology, syntax and discourse analysis. |
| Approach: | They treat projective dependency trees as latent variables in their probabilistic model and induce them in such a way as to be beneficial for a downstream task. |
| Outcome: | The proposed model is effective on sentiment analysis and natural language inference tasks and is fully differentiable. |
UniKER: A Unified Framework for Combining Embedding and Definite Horn Rule Reasoning for Knowledge Graph Inference (2021.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graph inference has been studied extensively due to its wide applications. |
| Approach: | They propose a framework that restricts logical rules to be definite Horn rules and can exploit the knowledge in logical rule-based reasoning and KGE in an extremely efficient way. |
| Outcome: | The proposed framework can exploit the knowledge in logical rules and improve KGE in an extremely efficient way. |
GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generation (2022.emnlp-main)
Copied to clipboard
Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A. Smith, Daniel Weld
| Challenge: | Effective human evaluation of text generation tasks remains an important, open area for research. |
| Approach: | They propose a system for running standardized human evaluations across different generation tasks. |
| Outcome: | The proposed system produces standardized human evaluations across tasks . it crowdsources predictions and ranks systems on leaderboards . the proposed system is not reproducible over time and different annotator populations . |
Annotating the Annotators: Analysis, Insights and Modelling from an Annotation Campaign on Persuasion Techniques Detection (2025.findings-acl)
Copied to clipboard
Davide Bassi, Dimitar Iliyanov Dimitrov, Bernardo D’Auria, Firoj Alam, Maram Hasanain, Christian Moro, Luisa Orrù, Gian Piero Turchi, Preslav Nakov, Giovanni Da San Martino
| Challenge: | Existing annotation campaigns based on heuristic guidelines have not been thoroughly discussed. |
| Approach: | They propose a probabilistic model for optimizing intervention scheduling to reduce the cost of an expert oversight in annotation tasks. |
| Outcome: | The proposed model advocates for an expert oversight in annotation tasks and periodic quality audits to reduce costs. |